Skip to content

feat: Implemented Bootstrap module in py-libp2p#711

Merged
seetadev merged 27 commits intolibp2p:mainfrom
sumanjeet0012:feature/bootstrap
Jul 25, 2025
Merged

feat: Implemented Bootstrap module in py-libp2p#711
seetadev merged 27 commits intolibp2p:mainfrom
sumanjeet0012:feature/bootstrap

Conversation

@sumanjeet0012
Copy link
Contributor

@sumanjeet0012 sumanjeet0012 commented Jun 29, 2025

What was wrong?

Bootstrap Module was not present in py-libp2p.
The py-libp2p library lacked a bootstrap discovery mechanism, which is essential for nodes to connect to predefined bootstrap peers when joining a network.
Issue #607

How was it fixed?

Implemented Bootstrap module in py-libp2p with the following key components:

1. Core Module Implementation

  • Created libp2p/discovery/bootstrap/ directory with:
    • bootstrap.py: Main BootstrapDiscovery class that processes bootstrap peer addresses
    • utils.py: Address validation and peer info parsing utilities
    • __init__.py: Module exports

2. Host Integration

  • Added bootstrap parameter to new_host() function in libp2p/__init__.py
  • Modified BasicHost and RoutedHost classes to support bootstrap discovery
  • Integrated with existing peer discovery event system

3. Key Features

  • Converts string multiaddresses to PeerInfo objects
  • Validates bootstrap addresses and handles errors gracefully
  • Adds discovered peers to peerstore with appropriate TTL
  • Emits peer discovery events for consistency with mDNS discovery
  • Prevents self-connections and duplicate peer handling

4. Examples and Tests

  • Created comprehensive example in examples/bootstrap/bootstrap.py with CLI interface
  • Added integration tests to verify functionality

To-Do

  • Clean up commit history
  • Add or update documentation related to these changes
  • Add entry to the release notes

Cute Animal Picture

put a cute animal picture link inside the parentheses

@sumanjeet0012 sumanjeet0012 changed the title feat: Implemented Bootstrap modulein py-libp2p feat: Implement Bootstrap modulein py-libp2p Jun 29, 2025
@sumanjeet0012 sumanjeet0012 changed the title feat: Implement Bootstrap modulein py-libp2p feat: Implement Bootstrap module in py-libp2p Jun 29, 2025
@sumanjeet0012 sumanjeet0012 changed the title feat: Implement Bootstrap module in py-libp2p feat: Implemented Bootstrap module in py-libp2p Jun 29, 2025
@seetadev
Copy link
Contributor

@sumanjeet0012 : Huge thanks for this fantastic contribution! 🙌

Very much appreciate the effort you've put into implementing the Bootstrap module — this fills a critical gap in py-libp2p by enabling nodes to reliably connect to predefined peers during startup. The modular structure, graceful error handling, and integration with the peer discovery system will enable us to productionize py-libp2p.

Also, thank you for taking the time to rebase the branch — once checked in, any remaining merge conflict issues can be resolved right away.

Looking forward to reviewing the PR in detail soon — and let us know if you need any help there.

.gitignore Outdated
Comment on lines +182 to +184
bootstrap_instructions.txt
.gitignore
README.md
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are these about? If a bootstrap_instructions.txt is being generated somewhere, we should delete it rather than ignoring it.

I don't think we'd ever want to ignore a .gitignore or README.md file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pacrob The .gitignore file was pushed accidentally.
I was trying to create documentations using AI and accidently pushed it.

@sumanjeet0012
Copy link
Contributor Author

sumanjeet0012 commented Jun 30, 2025

@pacrob I noticed that in js-libp2p, they include bootstrap node addresses such as /dnsaddr/bootstrap.libp2p.io/ipfs/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN.

Should py-libp2p also support resolving IP addresses from such DNS-based multiaddresses? If so, could you please guide me on how to implement this functionality?

I have checked the py-multiaddr repository and noticed that support for resolving DNS addresses has been recently added. However, I am unsure how to utilize this functionality within py-libp2p, as the resolve method does not currently appear to be available on the Multiaddress object.

@seetadev
Copy link
Contributor

seetadev commented Jul 1, 2025

@sumanjeet0012 :Great question — and really appreciate you digging into how js-libp2p handles bootstrap nodes. You're absolutely right: supporting DNS-based multiaddresses like /dnsaddr/bootstrap.libp2p.io/... is an important capability, especially for smoother cross-implementation compatibility and network discovery.

You're also spot-on about recent progress — the py-multiaddr repo now includes support for resolving DNS addresses, which is a key prerequisite. The next step is to hook that into py-libp2p's peer discovery and transport stack so we can resolve those multiaddrs at runtime.

Right now, as you noticed, the Multiaddr object doesn't yet expose a .resolve() method directly. One way to proceed would be:

  1. Use the updated multiaddr resolution utility — If the underlying logic is in place (e.g., via multiaddr.dns.resolve or similar), we can build a helper function in py-libp2p that:

    • Takes a multiaddr,
    • Checks if it's DNS-based (/dns, /dns4, /dns6, /dnsaddr),
    • Resolves it into one or more IP-based multiaddrs using the DNS resolution logic in py-multiaddr.
  2. Extend Multiaddr (optionally) — In the longer term, we could consider extending Multiaddr with a .resolve() method or a utility function that returns resolved addresses in a clean way — similar to how it's done in js-libp2p.

  3. Plug into dialer / peer discovery — Eventually, the resolver function can be called as part of the dialing logic when we attempt to connect to a peer that has a DNS-based address.

If you’re interested, we will be happy to help sketch out a small utility function to prototype this. Once we have that working, we could open a discussion or PR to integrate it more fully into the connection logic.

Really excited to see you exploring this — it’s exactly the kind of cross-impl insight that strengthens the ecosystem. Let’s definitely move this forward!

CCing @acul71 and @pacrob for their thoughts and feedback.

@seetadev
Copy link
Contributor

seetadev commented Jul 1, 2025

@sumanjeet0012 : Wish if you could add this to a new discussion page on bootstrap module. Me, @acul71 and @pacrob will discuss with you on py-multiaddrs there.

Wish to keep the PR easy to navigate for new developers interested in contributing or extending your nice work on bootstrap module in py-libp2p.

@acul71
Copy link
Contributor

acul71 commented Jul 2, 2025

I have checked the py-multiaddr repository and noticed that support for resolving DNS addresses has been recently added. However, I am unsure how to utilize this functionality within py-libp2p, as the resolve method does not currently appear to be available on the Multiaddress object.

Hello @seetadev @sumanjeet0012

DNS resolution examples for py-multiaddr.

This script demonstrates how to use DNS resolution functionality in py-multiaddr
with trio, showing how to resolve bootstrap node addresses like those used in js-libp2p.

## Overview

This script shows various examples of DNS resolution using py-multiaddr with trio:

1. **Basic DNS Resolution**: Simple resolution of DNS addresses to IP addresses
2. **Bootstrap Node Resolution**: Resolving real bootstrap node addresses with peer IDs
3. **DNS Protocol Comparison**: Testing different DNS protocols (/dns/, /dns4/, /dns6/, /dnsaddr/)
4. **Peer ID Preservation**: Ensuring peer IDs are maintained during resolution
5. **Sequential Resolution**: Processing multiple addresses sequentially
6. **py-libp2p Integration**: Example of how to use resolved addresses with py-libp2p

See here:
https://github.com/multiformats/py-multiaddr/blob/master/examples/dns/trio_dns_examples.py

@seetadev
Copy link
Contributor

seetadev commented Jul 2, 2025

I have checked the py-multiaddr repository and noticed that support for resolving DNS addresses has been recently added. However, I am unsure how to utilize this functionality within py-libp2p, as the resolve method does not currently appear to be available on the Multiaddress object.

Hello @seetadev @sumanjeet0012

DNS resolution examples for py-multiaddr.

This script demonstrates how to use DNS resolution functionality in py-multiaddr
with trio, showing how to resolve bootstrap node addresses like those used in js-libp2p.

## Overview

This script shows various examples of DNS resolution using py-multiaddr with trio:

1. **Basic DNS Resolution**: Simple resolution of DNS addresses to IP addresses
2. **Bootstrap Node Resolution**: Resolving real bootstrap node addresses with peer IDs
3. **DNS Protocol Comparison**: Testing different DNS protocols (/dns/, /dns4/, /dns6/, /dnsaddr/)
4. **Peer ID Preservation**: Ensuring peer IDs are maintained during resolution
5. **Sequential Resolution**: Processing multiple addresses sequentially
6. **py-libp2p Integration**: Example of how to use resolved addresses with py-libp2p

See here: https://github.com/multiformats/py-multiaddr/blob/master/examples/dns/trio_dns_examples.py

@acul71 : Great, thank you so much for sharing. This is indeed very helpful.

@sumanjeet0012
Copy link
Contributor Author

@acul71 Could you please try running the DNS resolution in py-libp2p using any example file?
It seems that the resolve method is currently functional only within the py-multiaddr repository. However, the recently implemented functions might not yet be usable in py-libp2p.

@acul71
Copy link
Contributor

acul71 commented Jul 7, 2025

@acul71 Could you please try running the DNS resolution in py-libp2p using any example file? It seems that the resolve method is currently functional only within the py-multiaddr repository. However, the recently implemented functions might not yet be usable in py-libp2p.

Since we don't still have access to py-multiaddr PyPi we have to find a way to use py-multiaddr not using dependency management via the standard PyPI mechanism (specifying the version in your project’s pyproject.toml or requirements.txt) requires an actual release.

  • I'm going to try this:
    Instead of specifying the dependency via PyPI, you can point to a specific commit or branch of py-multiaddr directly from GitHub in your pyproject.toml (if using Poetry or PDM) or requirements.txt (for pip).

@acul71
Copy link
Contributor

acul71 commented Jul 8, 2025

@sumanjeet0012 Py-libp2p Multihash Compatibility Fix #751

@acul71
Copy link
Contributor

acul71 commented Jul 10, 2025

@sumanjeet0012
This errors

ERROR tests/core/crypto/test_ed25519.py - AttributeError: module 'multihash' ...
ERROR tests/core/crypto/test_rsa.py - AttributeError: module 'multihash' has ...
ERROR tests/core/crypto/test_ed25519.py - AttributeError: module 'multihash' ...
ERROR tests/core/crypto/test_rsa.py - AttributeError: module 'multihash' has ...
ERROR tests/core/crypto/test_secp256k1.py - AttributeError: module 'multihash...
ERROR tests/core/crypto/test_secp256k1.py - AttributeError: module 'multihash...
ERROR tests/core/examples/test_examples.py - AttributeError: module 'multihas...
ERROR tests/core/examples/test_examples.py - AttributeError: module 'multihas...

should be caused by the new py-multiaddr, I think

Do you have errors locally?
Have you applied the patch for multihash (locally)?

@sumanjeet0012
Copy link
Contributor Author

@sumanjeet0012 This errors

ERROR tests/core/crypto/test_ed25519.py - AttributeError: module 'multihash' ...
ERROR tests/core/crypto/test_rsa.py - AttributeError: module 'multihash' has ...
ERROR tests/core/crypto/test_ed25519.py - AttributeError: module 'multihash' ...
ERROR tests/core/crypto/test_rsa.py - AttributeError: module 'multihash' has ...
ERROR tests/core/crypto/test_secp256k1.py - AttributeError: module 'multihash...
ERROR tests/core/crypto/test_secp256k1.py - AttributeError: module 'multihash...
ERROR tests/core/examples/test_examples.py - AttributeError: module 'multihas...
ERROR tests/core/examples/test_examples.py - AttributeError: module 'multihas...

should be caused by the new py-multiaddr, I think

Do you have errors locally? Have you applied the patch for multihash (locally)?

@acul71 I have applied the patch locally and two tests were failing locally I will try to fix them.
I think they are not directly related to multihash.
I will ping you once they are fixed.

@seetadev
Copy link
Contributor

@sumanjeet0012 and @acul71 : Nice collaboration and great efforts indeed. Appreciate the continued momentum.

Reviewing the PR in parallel as key changes are added.

@seetadev
Copy link
Contributor

@sumanjeet0012 : Wish to share that there are some merge conflicts coming up, which need to be resolved.

Seems like due to recent merge of PR: #766 (we updated py-multiaddrs).

CCing @acul71 .

@sumanjeet0012
Copy link
Contributor Author

@seetadev Resolved merge conflicts.
Ready for review.

@seetadev
Copy link
Contributor

@sumanjeet0012 : Appreciate your great work. Can you please share a small screencast or demo with bootstrap module? Kindly also include the updated developer docs at the discussion page. This is indeed a very important PR.

CCing @pacrob and @acul71: Hi Paul and Luca. Wish to have your feedback, pointers and review on this important PR, which enable us to use bootstrap module in py-libp2p.

@sumanjeet0012
Copy link
Contributor Author

@seetadev I have created a demo video demonstrating how bootstrap module is working in py-libp2p.
Demo video link: https://drive.google.com/file/d/1XAaA_gvTRXaFY5X-9csCw5n6BXx94tpT/view?usp=sharing

@seetadev
Copy link
Contributor

@sumanjeet0012 : Great, thank you Sumanjeet. This looks ready for final review + merge.

CCing @acul71, @lla-dane , @guha-rahul for a final feedback points.

@seetadev
Copy link
Contributor

@sumanjeet0012 : Reviewed the PR. Bootstrap module is indeed working fine. We will have some more improvements and integrations coming along soon. Doing a final review and merge.

Appreciate your efforts and initiative.

Also, wish to thank @acul71 for his efforts and support.

@seetadev seetadev merged commit 9b667bd into libp2p:main Jul 25, 2025
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants